A Multi-Sensor Approach for Activity Recognition in Older Patients

Existing surveillance systems for older people activity analysis are focused on video and sensors analysis (e.g., accelerometers, pressure, infrared) applied for frailty assessment, fall detection, and the automatic identification of self-maintenance activities (e.g., dressing, self-feeding) at home. This paper proposes a multi-sensor surveillance system (accelerometers and video-camera) for the automatic detection of instrumental activities of daily living (IADL, e.g., preparing coffee, making a phone call) in a lab-based clinical protocol. IADLs refer to more complex activities than self-maintenance which decline in performance has been highlighted as an indicator of early symptoms of dementia. Ambient video analysis is used to describe older people activity in the scene, and an accelerometer wearable device is used to complement visual information in body posture identification (e.g., standing, sitting). A generic constraint-based ontology language is used to model IADL events using sensors reading and semantic information of the scene (e.g., presence in goal-oriented zones of the environment, temporal relationship of events, estimated postures). The proposed surveillance system is tested with 9 participants (healthy: 4, MCI: 5) in an observation room equipped with home appliances at the Memory Center of Nice Hospital. Experiments are recorded using a 2D video camera (8 fps) and an accelerometer device (MotionPod®). The multisensor approach presents an average sensitivity of 93.51% and an average precision of 63.61%, while the vision-based approach has a sensitivity of 77.23%, and a precision of 57.65%. The results show an improvement of the multi-sensor approach over the vision-based at IADL detection. Future work will focus on system use to evaluate the differences between the activity profile of healthy participants and early to mild stage Alzheimer’s patients.